The purpose of face recognition systems is to identify a person from his picture by comparing an unidentified image, known as a probe image, to known images in a data base that has been prepared earlier.
The basic model of a face recognition system comprises two parts—Enrolment and Recognition (Verification/Identification). First, the initial registration “template”, which is a picture or collections of parameters, of the user (person to be identified) has to be constructed. This is done by collecting one or more sample images of his/her face. Salient features (for example, the coefficients of the Eigenfaces) are extracted from the samples, and the results are combined into the template. The construction of this initial template is called Enrollment. This initial template is then stored by the application (computer and software), and essentially takes the place of a password.
Thereafter, whenever the user needs to be authenticated, the user identifies himself, live images are captured from the camera, processed into a usable form, and matched against the template that was enrolled earlier. This form of authentication is called Verification, since it verifies that a user is who he says he is (i.e., verifies a particular asserted identity).
A second form of authentication is called Identification. In this form, a user does not have to assert an identity; the application compares the processed samples of the user against a specified population of templates, and decides which ones match most closely. Depending on the match probabilities, the user's identity could be concluded to be that corresponding to the template with the closest match.
Both in the enrollment and in the recognition stages, two important tasks must be carried out prior to the storage/recognition of the face—face detection and face normalization. The goal of face detection is to determine whether or not there are any faces in the image and, if present, notify the application of the image location and extent of each face. Given an image of a face, the goal of face normalization is to produce a canonical image that complies closely as possible with the given data base of face images.
Since normalization is used both in the enrollment and in the recognition stages, it should compensate for conditions, such as pose variations of the head or facial expressions, which might affect the appearance of a face in an image. A typical case is recognizing a non-frontal view of a face when the reference image in the database contains a frontal view of the same face. In this case the normalization algorithm is expected to produce a frontal view of the face so that it can be compared with the reference image.
The need to transform the probe image into a standard form is therefore obvious. But the most important question is—what is the optimal standard form? Normalization algorithms that have been published in the literature, as well as those used in commercial face recognition products, define the standard canonical form in advance. One example is rotating, scaling and translating the image so that the eyes appear at predefined locations. Another example is producing a frontal view of the face. These existing techniques disregard the original nature of the images to be entered into the database. For example, suppose the database images that were acquired by a camera mounted over the head with a 20 degree tilt angle. Instead of producing canonical frontal view faces prior to enrolling the images, it would be better to transform every new probe image to appear as if taken by a camera with a 20 degree tilt angle. A more complicated example is when all the people in the database appear smiling for some reason. In this case, better results can be expected if an image of a smiling face is produced prior to recognizing a new person.
It is a purpose of the present invention to provide a face normalization method that is best adapted to the original images to be enrolled into the database.
It is another purpose of the present invention to provide a face normalization method that best suites Eigenfaces methodology for recognizing faces.
It is yet another purpose of the present invention to provide a face normalization method that provides the best approximation of a face in the database with a constant template length; thereby providing the best recognition accuracy with the smallest storage capacity possible when using the Eigenfaces method.
Further purposes and advantages of this invention will appear as the description proceeds.